Rational-Based Visual Planning Monitors

The ability to act and respond to exogenous events in dynamic environments is crucial for robust autonomy. In dynamic environments, external changes may occur that prevent an agent from reaching its goal(s). I am interested in the design of reasoning and planning components operating in environments that undergo changes in real time. My goal is to develop a framework for fully integrated planning, execution and vision in dynamic environments. In the initial phase of this work, we have concentrated on the problem of enabling a planning system to deal with relevant changes in the environment during planning time. We introduce a new system for planning in a world under continuous change in an agent with visual perception. Our main contribution is to make vision sensitive to relevant changes in the environment that can affect an agent plans. We applied a rational-based monitor technique [Veloso et al., 1998] to the SHOP Hierarchical Task Network (HTN) planner [Nau et al., 1999]. We modified SHOP to generate plan monitors to interact with a vision system and react only to those environmental changes that bear on current planning decisions. Thus when the monitors detect any relevant changes, corresponding plan transformation are executed as needed. Rationale-based monitors provide a means of focusing visual attention on features of the world likely to affect the plan. When a feature being monitored changes, and the change is detected, we say that the monitor fires. Deliberation can then be performed to decide whether the plan under construction should be changed. If the planner decided to attend to the detected changes of the world state, it will perform a plan transformation. In particular, parts of the plan may be deleted because they have become unnecessary; new tasks may be added and current ones refined, and prior decisions about how to achieve particular goals may be revisited. Originally monitors were implemented in the state space planner Prodigy [Veloso et al., 1998], our work differs in using these monitors in the SHOP HTN planner. We have added our extended SHOP planner in the planning phase of a cognitive architecture named MIDCA [Cox et al., 2016]. The meta-cognitive, integrated dual-cycle architecture (MIDCA) consists of ”action-perception” cycles at both cognitive level and the meta-cognitive level. A cycle selects a goal and commits to achieving it. The agent then creates a plan to achieve the goal and subsequently executes the planned actions to make the domain match the goal state. MIDCA communicates with a Baxter humanoid robot to accomplish a goal in a dynamic environment using the monitors to focus vision and adapt plans. We have added an interface to MIDCA to communicate with ROS and the Baxter. It is responsible for sending messages to ROS as requested by MIDCA, and for placing messages received in appropriate queues for MIDCA to process. During the perceive phase, these messages will be accessed and stored in MIDCAs main memory. The interpret phase is responsible to reason about these messages and also create world states which are represented symbolically as logical predicates. Each monitor hires a perception node that is running asynchronously, which guides vision to focus on a specific features of the world. Additionally, we have run experiments in the blockworlds domain. Initial results show that planning with rationalebased monitors can reduce the total planning time when the world changes.